Enabling Environment Design via Active Indirect Elicitation

Many situations arise in which an interested party wishes to affect the decisions of an agent; e.g., a teacher that seeks to promote particular study habits, a Web 2.0 site that seeks to encourage users to contribute content, or an online retailer that seeks to encourage consumers to write reviews. In the problem of environment design, one assumes an interested party who is able to alter limited aspects of the environment for the purpose of promoting desirable behaviors. A critical aspect of environment design is understanding preferences, but by assumption direct queries are unavailable. We work in the inverse reinforcement learning framework, adopting here the idea of active indirect preference elicitation to learn the reward function of the agent by observing behavior in response to incentives. We show that the process is convergent and obtain desirable bounds on the number of elicitation rounds. We briefly discuss generalizations of the elicitation method to other forms of environment design, e.g., modifying the state space, transition model, and available actions.Engineering and Applied Science

Value-Based Policy Teaching with Active Indirect Elicitation

Many situations arise in which an interested party's utility is dependent on the actions of an agent; e.g., a teacher is interested in a student learning effectively and a firm is interested in a consumer's behavior. We consider an environment in which the interested party can provide incentives to affect the agent's actions but cannot otherwise enforce actions. In value-based policy teaching, we situate this within the framework of sequential decision tasks modeled by Markov Decision Processes, and seek to associate limited rewards with states that induce the agent to follow a policy that maximizes the total expected value of the interested party. We show value-based policy teaching is NP-hard and provide a mixed integer program formulation. Focusing in particular on environments in which the agent's reward is unknown to the interested party, we provide a method for active indirect elicitation wherein the agent's reward function is inferred from observations about its response to incentives. Experimental results suggest that we can generally find the optimal incentive provision in a small number of elicitation rounds.Engineering and Applied Science

Attendee-Sourcing: Exploring The Design Space of Community-Informed Conference Scheduling

Constructing a good conference schedule for a large multi-track conference needs to take into account the preferences and constraints of organizers, authors, and attendees. Creating a schedule which has fewer conflicts for authors and attendees, and thematically coherent sessions is a challenging task. Cobi introduced an alternative approach to conference scheduling by engaging the community to play an active role in the planning process. The current Cobi pipeline consists of committee-sourcing and author-sourcing to plan a conference schedule. We further explore the design space of community-sourcing by introducing attendee-sourcing -- a process that collects input from conference attendees and encodes them as preferences and constraints for creating sessions and schedule. For CHI 2014, a large multi-track conference in human-computer interaction with more than 3,000 attendees and 1,000 authors, we collected attendees' preferences by making available all the accepted papers at the conference on a paper recommendation tool we built called Confer, for a period of 45 days before announcing the conference program (sessions and schedule). We compare the preferences marked on Confer with the preferences collected from Cobi's author-sourcing approach. We show that attendee-sourcing can provide insights beyond what can be discovered by author-sourcing. For CHI 2014, the results show value in the method and attendees' participation. It produces data that provides more alternatives in scheduling and complements data collected from other methods for creating coherent sessions and reducing conflicts.Comment: HCOMP 201

Task Routing for Prediction Tasks

We study principles and methods for task routing that aim to harness people’s abilities to jointly contribute to a task and to route tasks to others who can provide further contributions. In the particular context of prediction tasks, the goal is to efficiently obtain accurate probability assessments for an event of interest. We introduce routing scoring rules for promoting collaborative behavior, that bring truthfully contributing information and optimally routing tasks into a Perfect Bayesian Equilibrium under common knowledge about agents’ abilities. However, for networks where agents only have local knowledge about other agents’ abilities, optimal routing requires complex reasoning over the history and future routing decisions of users outside of local neighborhoods. Avoiding this, we introduce a class of local routing rules that isolate simple routing decisions in equilibrium, while still promoting effective routing decisions. We present simulation results that show that following routing decisions induced by local routing rules lead to efficient information aggregation.Engineering and Applied Science

Crowdsourcing General Computation

We present a direction of research on principles and methods that can enable general problem solving via human computation systems. A key challenge in human computation is the effective and efficient coordination of problem solving. While simple tasks may be easy to partition across individuals, more complex tasks highlight challenges and opportunities for more sophisticated coordination and optimization, leveraging such core notions as problem decomposition, subproblem routing and solution, and the recomposition of solved subproblems into solutions. We discuss the interplay between algorithmic paradigms and human abilities,and illustrate through examples how members of a crowd can play diverse roles in an organized problem-solving process, serving not only as "data oracles" at the endpoints of computation, but also as modules for decomposing problems, controlling the algorithmic progression, and performing human program synthesis.Engineering and Applied Science